NEW FRAMEWORKS FOR EVALUATING 


COGNITIVE COMPLEXITY 


Frameworks for educators evaluating the cognitive complexity of 
assessment items in mathematics, science, and reading 


Throughout the country, state assessments in reading, mathematics, and science continue to play 
important roles in instructional improvement and accountability. State assessments must align to 
academic standards that are significantly more challenging than previous standards, reflecting 

the current knowledge and skill demands of postsecondary education and careers. They require a 
deeper understanding of subject matter, the ability to read more complex texts and materials, and 
the development and application of essential skills for collaboration, communication, inquiry, and 
problem solving. In short, new academic standards demand a more complex set of cognitive skills. 
Consequently, assessing student performance in these subjects is challenging and requires different 
kinds of assessments and assessment items. 


All statewide summative assessments must now require a range of cognitive demand. As Achieve has 
helped states design, develop, and evaluate state assessments, we’ve seen firsthand the challenge of 
assessing cognitively complex skills. Attempting to understand the cognitive complexity of assessment 
items is not new, but traditional frameworks now fall short because many are broad and content- 
agnostic. As new standards across all three content areas have become more reflective of the actual 
work students must do in each content area, understanding the cognitive complexity of the tasks that 
students complete requires content-specific frameworks. 


Achieve has developed three new frameworks - one each for mathematics, reading, and science - that 
can be used to evaluate the cognitive complexity of assessment tasks. Each framework is comprised of 
criteria and processes specific to its discipline and has been developed by content experts and practitioners. 
The frameworks are intended to help those who design, develop, and evaluate assessments ensure that 
the full depth and breadth of the state’s academic standards are being measured. 


Mathematics Framework 

Achieve’s new mathematics cognitive complexity framework allows for the evaluation of the cognitive 
complexity of individual mathematics assessment items through a simple two-step consideration of 
three aspects of rigor (conceptual rigor, procedural rigor, and application rigor). First, assessment 
reviewers apply Achieve’s Aspects of Rigor Matrix to an assessment item to determine its alignment to one or 
more of the three aspects of rigor. Second, after using the matrix to determine which aspect(s) of 
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rigor are targeted by an assessment item, the reviewer will then consider the complexity levels for each 
of the aspects of rigor targeted by that assessment item - assigning each aspect of rigor a complexity level 
of 1, 2, or 3 (according to the definitions provided in the framework). 


This process represents a new approach to evaluating mathematics assessment items against two of the 
Council of Chief State School Officers (CCSSO)’s 2013 mathematics-specific criteria for evaluating and 
procuring high-quality assessments: Assessing a balance of concepts, procedures, and applications and 
Requiring a range of cognitive demand. 


Reading Framework 


CCSSO’s 2013 criteria for evaluating and procuring high-quality assessments to accompany new college- 
and career-ready academic standards also included nine English language arts (ELA)/literacy-specific 
criteria. Achieve’s new framework proposes a new ELA/literacy-specific approach for measuring the 
cognitive complexity of reading items, drawing on the language of one of those criteria: Requiring a range 
of cognitive demand. 


The new framework takes into account research on the significance of text complexity in reading assessment 
items while also evaluating evidence and reasoning requirements. The framework is intended to allow 
users to determine the range of cognitive complexity present in reading assessments by completing an 
analysis of the individual reading items present in a test form or event. This analysis is then rolled up at 
both the passage/item set level and form level, allowing users to consider the complexity of the items on 
the entire form. With this approach, users can hone in on the sources of complexity in individual 
reading items to ensure that the needed range of complexity is present across the testing event. 


Science Framework 


Following the release of A Framework for K-12 Science Education and the Next Generation Science 
Standards, science education leaders are engaged in the difficult work of developing new assessments to 
measure learning under new three-dimensional science standards. New thinking about alignment 
between these standards and assessments requires new approaches for considering and evaluating 
cognitive complexity on assessment items and tasks. 


Achieve’s new science cognitive complexity framework offers a new approach to capturing and 
communicating the complexity of summative assessment items and tasks designed for three- 
dimensional standards. The complexity framework can be used to assess the degree to which an 
assessment task asks students to intellectually engage in and make use of disciplinary core ideas, science 
and engineering practices, and cross-cutting concepts in service of sense-making. The new framework 
allows for both the analysis of individual items or tasks and the holistic analysis of multi-component 
tasks or item clusters. 
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